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1. INTRODUCTION 


Fifteen Saskatchewan, Canada, test sites were processed by analysts In an 
operational mode during 1979. Landsat acquisitions of the sample segments 
were acquired In 1978 during the Large Area Crop Inventory Experiment (LACIE) 
Transition Year (TY). (Figure 1-1 shows the segment locations.) The segments 
were analyzed for "direct wheat" (i.e., spring wheat, oats, and flax) and 
barley. Direct wheat proportion estimates were produced on only 4 of the 
15 segments since the analysts determined that barley separation acquisitions 
were unavailable for the remaining 11 segments. 

The results and conclusions of two evaluations, a proportion estimation 
evaluation using ground-truth data and the labeling error characterization 
studies, are presented 1., sections 2 and 3, respectively. 

1.1 OBJECTIVES 

The objectives of this report are to present the evaluation of the segment 
proportion estimates as compared to the ground-truth proportion estimates and 
to provide detailed labeling error characterizations for a subset of the 
15 segments. 

1.2 SCOPE 

The proportion estimates for all 15 test sites were evaluated, but only 7 of 
the 15 segments were selected for detailed labeling error analysis and char- 
acterizations. Resource and time constraints did not permit a complete anal- 
ysis of all segments. 

The four segments with direct wheat estimates and three additional segments 
with relatively large proportion estimation errors were selected for labeling 
error characterizations. 
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Figure 1-1.- 1978 TY Canadian test site locations. 



2. PROPORTION ESTIMATION EVALUATION 


2.1 APPROACH AND METHODOLOGY 

The TY Canadian ground observation inventories are incomplete because of 
improper alinement of the ground-truth areas relative to the Landsat image of 
the sample segment. Consequently, segment proportion estimates were computed 
for ground observation areas common to both the original inventory and the 
Landsat image of the sample segment. The analyst estimates were calculated by 
using a random sample approach. This approach allowed analysts to label and 
evaluate all available grid dots in the ground- truthed area of a segment. Of 
the 209 available dots per segment, an average of 181 were studied by the 
analysts. This resulted in proportion estimates which could be evaluated 
against their corresponding ground observation estimates. The results of this 
evaluation are provided in table 2-1. 

The ground-observed proportion estimates were computed using approximately 
600 systematically selected ground-truth labels. Estimates of direct wheat, 
barley, and total spring small grains (which includes direct wheat and barley) 
were computed based on these selected dots. Two independent estimates were 
computed for each segment using the procedure described above, and the final 
proportion estimates were calculated using the average of the two. The 
results of the averaging are provided in table 2-2. 

The results in tables 2-1 and 2-2 were used in a study which compared the seg- 
ment proportion estimates with the ground-truth estimates. All 15 segments 
were evaluated for total spring small -grain accuracy. The four direct wheat 
segments were analyzed for barley separability, as well as for total spring 
small-grain accuracy. The results of these analyses are provided in 
table 2-3, 
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TABIE 2-2.- PROPORTION ESTIMATES DERIVED FROM 600-0OT 
GROUND TRUTH OF TY CANADIAN TEST SITES 











TABLE E-3.- PROPORTION ESTIMATION ERROR OF 
TY CANADIAN TEST SITES 


[Analyst proportion estimate; minus 
ground- truth proportion*] 



Direct wheat, 
percent 

Barley, 

percent 

Spring 

small grains, 
percent 

30S0 

-8.6 

7.6 

-1.1 

3053 

-1.0 

1.8 

.9 

3064 

• 


-3.5 

3080 



.4 

3083 



4.1 

3093 



-2,8 

3112 

-4.4 

2.8 

-1.7 

3132 



-1.9 

3163 



-8.7 

3166 

■ 


-6.7 

3169 



3.9 

3175 



-6.2 

3185 

1.1 

-6.4 

-5.2 

3197 



-7.8 

3201 



-7.3 

Results: 




Mean error 

-3.2 

1.5 

-2.9* 

Standard 

deviation 

4.3 

5.8 

4.1 

Mean 

squared 

error 

24.0 

27.2 

24.1 

Number of 
segments 

4 

4 

15 


*The value of the t-test statistic = -2.7, which 
indicates that the average difference between 
analyst spring small -grain proportion estimates 
and ground- truth proportion estimates is signifi- 
cantly different from zero at the 10-percent level. 
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2.2 RESULTS 




2.2.1 LAC IE PHASE III/TY COMPARISON 

2.2. 1.1 Soring Small -Grain Proportion Estimation Comparison 

The TY Canadian processing resulted in a mean error of -2.9, a standard error 
of 1.9, and a relative difference (RO) of -6.8 percent for spring small -grain 
proportion estimation accuracy. These results compare favorably with the U.S. 
northern Great Plains (USNGp) results for the TY, where the mean error was 
-0.8, the standard error was 1.0, and the RD equaled -3.9 percent. The TY 
Canada results were significantly better than the LACIE Phase III results for 
the USNGP (which were -6.1, 0.8, and -17.5 percent for the mean error, stand- 
ard error, and RD, respectively). 

The LACIE Phase III and TY results for the USNGP were computed using the 
400-dot ground-truth proportion estimation technique. See table 2-4 for a 
comparison of the data described above. 

2. 2. 1.2 Direct Wheat/Barley Proportion Estimation Comparison 

The Canadian TY direct wheat and barley proportion estimation was computed 
from a very small sample (four sample segments). Because of this small number 
of estimates, definite conclusions regarding these results are not possible. 
The mean error for the TY direct wheat and barley estimates of -3.2 and 1.5 
are comparable to the TY USNGP results of -0.7 and -0.1 for direct wheat and 
barley, respectively. The TY Canadian result is also comparable to the USNGP 
LACIE Phase III direct wheat result, which has a mean error of 0.5. Direct 
barley was not computed during Phase III; therefore, a comparison cannot be 
made. Table 2-5 contains comparative results for the USNGP Phase III, USNGP 
TY, and Canadian TY propo*"tion estimation accuracies. 
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TABLE 2-4.- COMPARISON OF SPRING SMALL-GRAIN PROPORTION 
ESTIMATION ACCURACIES 


Statistic 

LACIE Phase III 
USNGP 

TY 

USNGP 

TY 

Canada 

n 

45 

45 

15 

^SG 

34.9 

20.5 

42.5 

IT 

-6.1 

-.8 

-2.9 


.3 

1.0 

1.9 

RD 

-17.516 

-3.9% 

-6,8% 


TABLE 2-5.- COMPARISON OF DIRECT WHEAT/BARLEY PROPORTION 
ESTIMATION ACCURACIES 


LACIE Phase III 
USNGP 

Statistic (North Dakota only) 


20 

25.1 

.5 

1.8 

2 . 0 % 


Symbol definitions: 

n » number of sample segments 
^SG " 9 >^ound-truth proportion (400-dot) 
U ■ mean error 
a-jj » standard error 


TY 


TY 


USNGP 


Canada 


Direct wheat 

Barley 

Direct wheat 

Barley 

45 

45 

4 

4 

17.6 

2.9 

50.6 

4.0 

•• 7 

-• 1 

-3.2 

1.5 

1.1 

1.5 

2.2 

2.9 

-4.0% 

-3.5% 

-6.3% 

37.5% 
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2.2.2 CORRELATION STUDY OF SPRING SMALL-GRAIN ESTIMATES 


Plots of the spring small -grain test site data in tables 2-1 and 2-2 are shown 
in figure 2-1. The data points show that there is approximately a 3-percerit 
negative bias of analyst dots, with 11 of the 15 points below the perfect cor- 
relation line (45* diagonal). The correlation coefficient for these data is 
r * 0.956, The value of the t-test statistic is -2.7 (RD « -6.4 percent), 
which indicates that the average difference between analyst spring small-grain 
proportion estimates and ground-truth proportion estimates is significantly 
different from zero at the 10-percent level. This result is comparable to the 
TY proportion estimation performance in the USNGP. 

2.2.3 TREND ANALYSIS 

Plots of the proportion estimation error versus the ground-truth proportions 
for the spring small grains in the Canadian test sites are shown in fig- 
ure 2-2. It does not appear that the trend for the error is related to the 
proportion estimate value as in the U.S. Great Plains wheat segments, where 
larger negative errors were associated with larger proportion estimates. 

(See figure 4-6 in reference 1, page 4-32.) 

2.2.4 DISTRIBUTION OF SPRING SMALL-GRAIN PROPORTION ESTIMATION ERRORS 

The plot in figure 2-3 indicates that a bimodal distribution of spring small - 
grain errors exists. The dashed line in the figure approximates the division 
between the two distributions. The six segments with, the larger proportion 
estimation errors (3163, 3166, 3175, 3185, 3197, and 3201) and segment 3169 
are the northernmost segments processed. The RD for these seven northernmost 
segments is -11.2 percent, whereas the southernmost segments have a much 
smaller RD of -1.6 percent. Except for segment 3169, the northernmost seg- 
ments had poor accuracies, due partially to differences in agricultural crop- 
ping practices that exist between the area containing these segments and those 
in the southernmost area. The southernmost segments, where spring wheat and 
barley are the primary crops, generally have less complex cropping practices. 
In the northernmost segments, rapeseed, corn, and potatoes are grown along 
with small grains. It is believed that the more complex cropping system 
increased the number of analyst decisions, which in turn increased the oppor- 
tunity for labeling errors to occur. 


Figure 2- 
analyst 


Proportion estimate from 600-dot ground truth, percent 


Plot of spring small -grain proportion estimates derived from 
labels versus proportion estimates derived from ground truth. 
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Analyst proportion estlaate Minus 
ground-truth proportion estlnate. percent 



1 ) 


-9' 
-8 ' 
-7 
-6' 
-5' 
-4- 


2 -3- 
o 
u 
u, 

0 ) . 2 ' 


Segment 

number 


p"31 63 

=^3197 

n =• 7 (including 3169) 

^3201 

d » -4.69% 

—3166 

—3175 

RD » -11.2% 

Northernmost test sites 

—3185 

(including 3169) 

▲ 


(ts 

■M 

O 


•1 ■ 
0- 
1- 


•3064 

•3093 

■3132 
•31 1 2 

•3050 


■3080 

■3053 


Southernmost test sites 
(except 3169) 

n » 8 
Z = -0.69% 

RD = -1.6% 



Symbol definitions: 

n = number of segments 
d = average difference 
RD = relative difference 


Figure 2-3.- Plot of spring small -grain proportion estimation error for the 
northernmost versus the southernmost segments. A distinctive break in 
accuracy (dashed line) is evident between the two groups of segments. 
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The differences In these two areas, northernmost and southernmost segments, 
are evident from the results of the detailed labeling accuracy study from 
which this conclusion was derived. These results are discussed in sec- 
tion 3.2.1. 

2.3 CONCLUSIONS 

The analysis of the analyst-derived proportion estimates compared with the 
600-dot ground-truth estimates shows a small underestimation (RD » -7.0 per- 
cent) of spring small grains. This underestimation is statistically signifi- 
cant (the value of t-test » -2.7), but the results are better than those for 
LACIE Phase III in the U.S. spring wheat states and are comparable to the TY 
results in the same area. 

The direct wheat results indicate an underestimation of direct wheat 
(RD « -6.4 percent) but an overestimation of barley (RD » 36.1 percent). It 
should be pointed out that these results are for only four segments, which is 
too few a number to arrive at any definite conclusions. 

Proportion estimation accuracy was better for the southernmost Saskatchewan 
segments (RD = -1.6 percent). The poorer proportion estimation accuracy for 
the northernmost segments (RD = -11*2 percent) is believed to result from the 
cropping system in this region, which is generally more complex than in the 
southernmost area of Saskatchewan. It has been concluded that the more com- 
plex cropping system resulted in more decision points for the analyst, which 
in turn increased the potential for labeling errors. 


3. LABELING ERROR CHARACTERIZATION STUDIES 


3.1 APPROACH AND METHODOLOGY 

Seven segments, which include the four barley separation segments and three 
additional segments with relatively large spring small -grain proportion esti- 
mation errors, were evaluated for spring small -grain and direct wheat accu- 
racies. The evaluation studies compared the analyst labels against ground 
truth. This resulted in a numerical tabulation of the errors, both omission 
and commission, and a characterization of these errors. 

For the four direct wheat segments, ,in the analysis, three classes of labels 
were measured against ground truth; i.e., barley, direct wheat, and all other 
crops as a single group labeled as nonspring small grains. These four seg- 
ments also were analyzed along with the three additional segments as a single 
group for two-class accuracy; i.e., spring small grains and nonspring small 
grains. In addition, the two groups of segments (four direct wheat and three 
additional) were analyzed separately for their two-class accuracies. 

All analyst labels, for both type 1 and 2 dots, were grouped and evaluated 
together, rather than separately as in previous accuracy assessment 
evaluations. 

3.2 TWO-CLASS ANALYSIS OF SEVEN SEGMENTS 

The results of tabulating the number of spring small -grain errors for each of 
the seven segments are shown as confusion matrices in table 3-1. The matrices 
are arranged such that the four segments processed for direct wheat are on the 
left [tables 3-1 (a) through (d)] vind the three additional segments are on the 
right [tables 3-l(e) through (g)]. 

The percentage of correctly labeled picture elements (pixels) was quite high 
and relatively close for all segments except one. The percentage of correctly 
labeled spring small grains varied for all segments, from approximately 
91 percent to 98 percent, except for segment 3197 which had a relatively low 
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TABLE 3-1.- TWO-CLASS SPRING SMALL-GRAIN ACCURACIES 
FOR ALL SEVEN SEGMENTS 


Analyst -derived direct Mheat 
estlniates 


(a) Segment 3050 


Additional segments selected for 
error characterizations 


(e) Segment 3163 


®^SSG I? • ^ - 0.037 


^0 ?7' 


0.023 


II - 0.977 


STsscTI?- ^-0.019 


n?" ° 


119 - 1 nn 

Yjg’ ■ 1.00 


(b) Segment 3053 


(f) Segment 3169 


‘'^SSG W " T§r “ 


'O 77" 


0.026 


II - 0.974 


®^SSG H • 4 • 0-051 


0 W 


0.088 


W “ 0.912 


(c) Segment 3112 


(g) Segment 3197 


O^SSG TUT " 0*980 -j||- ■ 0.020 


GTo ^- 0.046 11 - 0.954 


O^SSG 1^- 0.674 II - 0.326 


0 ITT" 


0.008 


III ■ 8*992 


(d) Segment 3185 


STssg ® ■ 0-S06 31-0.094 


'O W 


0.021 


^• 0.979 , 


Symbol deflriitions.’ 

AIssg ■ analyst- Interpreter propor- 
tion estimate of spring 
small grains 

AIq ■ analyst- Interpreter propor- 
tion estimate of nonspring 
small grains 

OTssg ■ ground -truth proportion 
estimate of spring 
small grains 

GTg - ground-truth proportion 
estimate of nonspring 
small grains 







accuracy of 67 percent; The low labeling accuracy for segnwnt 3197 can be 
attributed to the analysts being overly concerned with separating rapeseed 
from spring small grains and inadvertently mislabeling 12 spring small -grain 
dots. In this report, these errors are referred to as "oversights," which 
accounted for 80 percent of the omission errors in this segment and 34 percent 
of all omission errors in the seven segments evaluated. 

The confusion matrix for omission and commission errors for the seven segments 
treated as a group is given in table 3-2. The percentage of correctly labeled 
spring small-grain dots is approximately 94 percent, which can be considered 
as excellent performance. This result exceeds the TY U.S. spring wheat states 
accuracy of approximately 70 percent by a considerable amount i however, a num- 
ber of factors unique to Canada may account for this difference. 

The excellent spring small-grain labeling accuracy may be attributed primarily 
to three major factors: the data set, the uniqueness of Saskatchewan agricul- 
tural practices, and the segment processing approach. The data set was 
thoroughly screened to eliminate segments with Inadequate and marginal acqui- 
sition histories. The intent was to select segments which had acquisitions in 
all four major biowindows, and this is believed to have contributed to the 
improved labeling accuracy. 

The uniqueness of agricultural practices in Saskatchewan was recognized during 
the segment processing when ancillary data inputs indicated that almost exclu- 
sively spring crops, including spring small grains, are grown in this prov- 
ince. The ancillary statistics indicated the presence of very few summer 
crops; i.e. , beans and potatoes. For all of Saskatchewan, beans and potatoes 
combined amounted to 0.02 percent of all crops sown in 1971. This implies 
that limited confusion could be expected from these crops. The ancillary data 
and previous years' intensive test site data showed that rapeseed was the only 
other major spring crop. 

The implication of the above factors is that the only major potential confu- 
sion crop is rapeseed; therefore, if a crop seems to emerge in the spring, 
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TABLE 3-2. > TWO-CLASS SPRING SMALL-GRAIN ACCURACIES 
FOR ALL SEVEN SEGMENTS COMBINED 



^^SSG 


AIq 

®^SSG 

Uy ■ 0-M6 

35 

m 

- 0.064 

STfl 

_y^ • 0.028 

648 

661 

- 0.972 


3-4 


iri '-'U . ^ 




from May through June» it is probably either a spring small grain or rapeseed. 
The qualifier to this is that the spring-emerged crops must, after emergence, 
progress through a multitemporal signature which includes at least turning and 
harvest or postharvest signatures* This identification technique is consist- 
ent with the TY detailed inalysis procedures (see ref. 2, appendix C, sec- 
tion C.l). 

The rapeseed was readily separated from the spring small grains because of its 
distinctive bright pink/purple signature. It was concluded that, once the 
rapeseed was separated, the remaining spring crops were most likely to be 
spring small grains. The labeling results show that this decision process 
was satisfactory. 

The steps described above were enhanced by the Canadian processing technique 
which emphasized the team approach, whereby labeling decisions to be made were 
discussed thoroughly by a team. Since only 15 segments were Involved in the 
processing, it was possible to utilize this approach to advantage. 

The team attempted to maximize the use of ancillary data (e.g., crop calen- 
dars, Saskatchewan census subdivision statistics, and meteorological data) and 
intensive test site data from LACIE Phase III. In addition, since only 15 
segments were being processed, the analyst timeline was increased to 2.5 man- 
days per segment to ensure adherence to proper analysis procedures. This com- 
pares to the LACIE Phase III timeline of approximately 1.5 man-days per seg- 
ment for analyst processing, quality assurance, and operations verification. 
The latter should not be confused with the analyst processing timeline (a 
subset of the 1.5 man-days) which was 3 to 4 hours during Phase III. 

3.2.1 DETAILED ERROR CHARACTERIZATIONS FOR SPRING SMALL GRAINS 

The labeling error characterizations are summarized by segment in table 3-3. 
The total numbers of errors of omission and commission for the seven segments 
are 35 and 19, respectively. Oversights or mislabeling without any observable 
reason accounted for the majority of the errors, 22 errors (20 omission and 2 
commission). The next largest error was caused by border/edge dot confusion. 


3-5 


TABLE 3-3.- DETAILED SPRING SMALL-GRAIN ERROR OIARACTERIZATION 
















11 errors (5 omission and 6 commission). The third largest error was caused 
by omission errors resulting from grass/pasture confusion, seven errors. 

Summer crop confusion of beans and potatoes resulted In four commission 
errors, all four of which were In segment 3169. Although the bean and potato 
fields emerged later than the majority of the spring small grains, their 
lateness was not considered to be outside the range for small grains predicted 
by the normal crop calendars. An evaluation of these fields Indicates that 
possibly they may be separated from the spring crops because of the later 
green-up or emergence and bright red signatures; but, this will probably 
result In omission of some flax ar'< spring wheat which exhibit similar 
signatures. 

The remainder of the omission and commission errors resulted from confusion of 
rapeseed, winter rye, and idle fallow with residue. It has been concluded 
that only the oversights (22 errors) could have been labeled correctly and 
that the labeling errors resulting from border/edge confusion (11 errors) and 
misidentificatlon (21 errors) were justifiable. The implication of this anal- 
ysis is that if the oversight errors had not occurred there would have been 15 
omission errors (2.2 percent) and 17 commission errors (3.1 percent), result- 
ing In an unbiased spring small -grain proportion estimate for these seven 
segments. 

3.2.2 DISTRIBUTION OF SPRING SMALL-GRAIN LABELING ERRORS 

The proportion estimation evaluation (section 3.2) indicated that the north- 
ernmost Saskatchewan segments generally have higher estimation errors than the 
southernmost segments. This is supported by the detailed error characteriza- 
tion, which shows a total of 40 labeling errors of omission and commission for 
the four northernmost segments evaluated and only 14 errors for the three 
southernmost segments. The distribution of labeling errors by segment is 
shown In figure 3-1. 

As stated previously, the reason for the high estimation errors for the north- 
ernmost segments is that the more complex cropping practices in this region 
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Total errors of omission and commission 


Segment 
. number 


■3169 


16-T-3197 
15 
14 ' 

13 ’ 

12 
11 
10 
9 ' 


•3185 



Northernmost seoments 
(including 3163) 


Southernmost segments, 
(except 3163) 


Figure 3-1.- Distribution of labeling errors in the seven segments. 
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increased the number of analyst decisions, which in turn increased the oppor- 
tunity for the occurrence of labeling errors. 

3.3 TWO-CLASS ANALYSIS OF FOUR DIRECT WHEAT AND THREE ADDITIONAL SEGMENTS 

AS 'SEPARATE GR'PDF? 

The 96-percent correctly labeled spring small grains for the four direct wheat 
segments [table 3-4(a)] is significantly better than the 89-percent accuracy 
for the three additional segments [table 3-4(b)]. The difference in the two 
results is the relatively low labeling accuracy of segment 3197, which is 
included in the three-additional -segments grouping. 

3.4 THREE-CLASS ANALYSIS OF THE FOUR DIRECT WHEAT SEGMENTS 

The results of tabulating the number of errors for each of the four direct 
wheat segments (3050, 3053, 3112, and 3185) are shown as confusion matrices in 
tables 3-5(a) through (d). Table 3-6 contains the confusion matrix for the 
four segments combined. 

The percentages of correctly labeled direct wheat and barley are approximately 
89 and 31 percent, respectively (table 3-6). The results clearly indicate 
that barley could not be separated consistently from the other spring small 
grains in Saskatchewan using the barley separation procedure (ref. 2, appen- 
dix C, section C.1.2.2). Barley was primarily confused with spring wheat; a 
minor amount was confused with rapeseed and grass/pasture. The result for 
labeling direct wheat is rather good, approximately 89 percent correctly 
labeled, compared to the TY labeling accuracy of approximately 70 percent for 
the U.S. spring wheat states. 

Barley incorrectly labeled as spring wheat accounted for 14 errors, while 
barley labeled as nonspring small grains amounted to 4 errors (table 3-6). 

In addition, 26 spring wheat ground-truth pixels were incorrectly labeled as 
barley. 

The confusion occurred when barley was incorrectly labeled as direct wheat on 
the predicted barley separation acquisition. The barley was turning and being 
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TABLE 3-4.- TWO-CLASS SPRING SMALL -GRAIN ANALYSIS 


(a) Four direct wheat segments 
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TABLE 3-5.- THREE-CLASS ANALYSIS OF THE FOUR DIRECT WHEAT SEGMENTS 
[Spring wheat/barley separation] 
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Segment 3050 
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Segment 3053 
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(c) 

Segment 3112 




(d) 

Segment 3185 
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Symbol definitions: 

Alg ■ analyst-interpreter proportion estimate of 
barley 

AIs ■ analyst-interpreter proportion estimate of 
spring wheat 

GTg • ground-truth proportion estimate of bark-y 

GT5 • ground-truth proportion estimate of spring 
^eat 


TABLE 3-6.- THREE-CLASS. ANALYSIS OF THE FOUR 
DIRECT WHEAT SEGMENTS COMBINED 

[Spring wheat/barley separation] 
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harvested along with the majority of the direct wheat. These fields were 
labeled correctly according to the direct wheat procedure (ref. 2, appen- 
dix C, section C.1.2.2); however, since the barley did not ripen before spring 
wheat as expected, it was omitted and labeled as direct wheat. The normal 
crop calendar used for the analysis of segment 3050, which had considerable 
barley/spring wheat confusion, is shown In figure 3-2. The crop calendar 
Indicates that barley should normally ripen before spring wheat, but potential 
confusion Is evident since the flax and oats crop calendars for the rlpe-to- 
harvest period are similar to that for barley. 

The incorrect labeling of spring wheat as barley occurred because of the early 
ripening signature for spring wheat on the barley separation acquisition. 
Procedurally, these fields were also labeled correctly. The problem appar- 
ently is because varieties of spring wheat and barley behave differently than 
those used In the development of the direct wheat procedure. 

Of the four barley fields labeled as nonspring small grains, two were narrow 
fields in segment 3185. These fields were near the edge of a lake and there- 
fore were interpreted as grass/pasture. Also in segment 3185, two barley 
fields were Interpreted, to be rapeseed because of the distinctive bright 
purple/pink signature on the August acquisition. Rapeseed correctly labeled 
as nonspring small grains in segment 3185 exhibited somewhat similar signa- 
tures on the same August acquisition; therefore, the confusion between barley 
and rapeseed is understandable. It is possible that a different or more com- 
plete data set may have provided information to avoid this confusion. 

3.5 CONCLUSIONS 

Approximately 94 percent of the spring small grains for the seven Saskatchewan 
segments were labeled correctly, which can be considered as excellent perform- 
ance. This result exceeds the TY U.S. spring wheat states labeling accuracy 
of 70 percent. 

The direct wheat labeling accuracy was good, approximately 89 percent, despite 
the low barley separation accuracy of 30 percent. This compares to the TY 
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Figure 3-2.- Southern Saskatchewan crop calendar used in the analysis 
of segment 3050. A dashed line indicates an estimate. 
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U.S. spring wheat states labeling accuracies of 70 and 65 percent, 
respectively. 

The high spring small -grain labeling accuracy of 94 percent may be attributed 
to three major factors: a good data set with adequate acquisition histories, 
few confusion crops In Saskatchewan, and a processing technique which empha- 
sized the team approach In all aspects of the analysis procedures. 

The detailed characterization of the spring small-grain errors shows that the 
largest sources Of errors are analyst oversights, border/edge dot confusion, 
and grass/pasture confusion. From the analysis. It can be concluded that the 
spring small-grain labeling accuracy could have been higher If the oversight 
omission and commission errors had not occurred. 

Labeling errors, both omission and commission, were higher in the northernmost 
segments than In the southernmost segments. It has been concluded that this 
Is because of the more complex cropping system in the region occupied by these 
segments. It is believed that the more complex cropping practices resulted in 
more analyst decisions, which increased the opportunity for the occurrence of 
labeling errors. Many of these errors were those designated as "oversight 
errors." 

The results show that barley separation was largely unsuccessful in the 
Canadian test sites. A better understanding of the development of barley and 
spring wheat in Saskatchewan is required in order to develop a procedure to 
separate barley from the other spring small grains in this region. Research 
Into these problems should continue. 
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